The present invention relates generally to the field of question answering systems, and more particularly to evaluating questions asked of question answering systems.
Question answering (QA) is a known computer science discipline within the fields of information retrieval and natural language processing (NLP). Generally speaking, QA systems automatically answer questions posed by humans in natural language, typically by either querying a structured database of knowledge/information or by retrieving answers from unstructured collections of natural language documents. QA systems are commonly designed to answer questions about a specific set of topics (also referred to as a “domain”), where a question pertaining to the QA system's domain is considered “on-topic,” and a question that does not pertain to the QA system's domain is considered “off-topic.”
Named entity recognition (NER) is known. NER typically involves detecting and classifying in-text elements into pre-defined categories, such as names, dates, locations, and quantities, for example.